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Abstract 

The purpose of this study was to identify the effect of scripted roles on students’ level of 
cognitive presence in asynchronous online threaded discussions. A quantitative content analysis 
was used to investigate: (1) what level of cognitive presence is achieved by students’ assigned 
roles in asynchronous online discussions; (2) differences between students’ cognitive presence 
when the asynchronous online discussions occur during a 5-week intensive summer class versus 
a 15-week regular class (fall and spring); and (3) the impact of the types of questions on 
students’ cognitive presence in role-based asynchronous online discussions across three 
semesters in an online introductory nutrition course. The participants in this study were 139 
undergraduate students at a major public university in the mid-Atlantic region of the United 
States. The results of this research correspond to the findings of previous research that indicate 
scripted roles can be an effective strategy to improve both learning processes and outcomes. In 
addition, this study found differences in students’ level of cognitive presence (exploration and 
integration) based on the course length. Finally, this study found evidence that the types of 
questions asked related to the level of cognitive presence, i.e., higher level questions can lead to 
higher level of cognitive presence and vice versa. 

Keywords : asynchronous online discussions; role assignment; scripted roles; cognitive presence; 

online teaching and learning. 
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Introduction 

As online courses and programs have increased, the online discussion board has become 
one of the most important spaces for knowledge construction through communication and social 
interactions between students and instructors in asynchronous online learning (Xie, Miller, & 
Allison, 2013; Xie, Yu, & Bradshaw, 2014; Wise & Chiu, 2012). Multiple studies have 
demonstrated the effectiveness of asynchronous online learning environments for knowledge 
acquisition by undergraduates in nutritional sciences, but effectiveness of discussion boards 
specifically were not addressed (Cohen, Carbone, & Beffa-Negrini, 2011; Franko, Cousineau, 
Trant, Green, Rancourt, Thompson, & Ciccazzo, 2008). In order to make decisions based on 
complex knowledge, students in nutritional science courses must achieve a high level of critical 
thinking to analyze and synthesize information from various sources. Therefore, to help students 
achieve a higher level of critical thinking in nutritional sciences, researchers and practitioners of 
online learning have explored the effectiveness of different instructional strategies including role 
assignment (De Wever, Van Keer, Schellens, & Valcke, 2010; Wise, Saghafian, & 
Padmanabhan, 2012). The implementation of role-based discussions encourages students to 
practice critical thinking in a structured, supervised enviromnent, and achieve cognitive presence 
necessary to construct a meaningful response to complex nutritional questions (Cohen et ah, 
2011). In this study we define roles as stated functions and/or responsibilities that guide students’ 
behavior and group interaction (Strijbos & Weinberger, 2010, p.491). In addition, cognitive 
presence is defined as “the extent to which learners are able to construct and confirm meaning 
through sustained reflection and discourse in a critical community of inquiry” (Garrison, 
Anderson, & Archer, 2001, p. 11). However, simply assigning roles in asynchronous online 
discussions does not guarantee that students can reach higher levels of cognitive presence 
(Garrison et ah, 2001). 

Studies also investigated other instructional strategies such as the type of question 
prompts that might have an impact on students’ cognitive presence in asynchronous online 
discussions. Studies revealed that higher level question prompts such as problem-based prompts 
or case-based prompts are likely to lead to higher levels of cognitive presence (deNoyelles, 
Zydney, & Chen, 2014; Richardson, Sadaf, & Ertmer, 2012). Considering that improving 
cognitive thinking ability itself has become an important learning outcome which needs to be 
achieved through discussion activities, it would be necessary to explore the effects of the 
instructional strategies including role assignment and the types of question prompts on the levels 
of students’ cognitive presence in asynchronous online discussions. 

For this reason, the authors of this study explored the effects of the role assignments, 
specifically, scripted role assignments, on the levels of students’ cognitive presence in 
asynchronous online discussions of students in a nutrition science course. Scripted role 
assignments are usually assigned by the instructor and involve a single weekly task, as they were 
originally developed to pre-structure asynchronous online discussions (Strijbos & De Laat, 
2010). In this study, each student was assigned a specific task: starter, skeptic, or wrapper. A 
starter was responsible for posting a preliminary response to get the discussion started. A 
wrapper summarized the points that were made during the discussion. A skeptic challenged the 
points made by other students. This study also explored the effects of the types of question 
prompts on students’ level of cognitive presence in role-based asynchronous online discussions. 
For this study, the following types of questions were identified: (1) article summary, where 
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students were asked to read and summarize an article; (2) debate, where students were required 
to find a scientific reference and a non-scientific reference to support their assigned topic; (3) 
design intervention, where students were asked to design an intervention to promote physical 
activity; (4) design study, where students practiced the scientific method and got additional 
practice at identifying strengths and weaknesses in a research study design; (5) scenario-based, 
where students were provided with scenarios to analyze based on provided criteria; and (6) 
search and match, where students searched for resources and matched them with a supplied list 
of criteria. Therefore, the research questions in this study were: 

RQ#1: What level of cognitive presence is achieved by students’ assigned roles in 
asynchronous online discussions? 

RQ#2: Are there any differences between students’ cognitive presence when the 
asynchronous online discussions occur during 5-week intensive summer classes versus 15-week 
regular classes (fall and spring)? 

RQ#3: What is the impact of the types of questions on students’ cognitive presence in 
role-based asynchronous online discussions across three semesters? 

Literature Review 

Post-Secondary Nutrition Science Education 

Nutrition is the science of foods and their nutrients and their actions within the body, and 
the application of this knowledge is used to modify dietary intake to optimize health outcomes. 
Given the current obesity crisis (Flegal, Kruszon-Moran, Carroll, Fryar, & Ogden, 2016; Ogden 
Carroll, Lawman, Fryar, Kruszon-Moran, Kit, & Flegal, 2016), it has become increasing clear 
that certain environmental constraints raise the risks for obesity (Myers, Slack, Martin, Broyles, 
& Heymsfield, 2016), and that there is a remarkable amount of nutritional misinformation 
circulating in popular culture (Korownyk, Kolber, McCormack, Lam, Overbo, Cotton, & Allan, 
2014; Wansink, 2005). Nutritional knowledge has thus become essential for maintaining a 
healthy lifestyle. Nonetheless, many young adults reach college age without adequately 
developing this life skill, and nearly 20% are obese by the time they reach adulthood (Ogden et 
al., 2016). Introductory nutrition courses are common on college campuses as a required course 
for students entering health-based disciplines or as a science or health elective. Research has 
already demonstrated the effectiveness of course-based nutrition knowledge on health outcomes 
and its predictors, such as body weight (Matvienko, Lewis, & Schafer, 2001). 

Cognitive Presence in Online Discussions 

To explore the level of deep cognitive thinking in learning, many researchers have 
focused on cognitive presence (Akyol & Garrison, 2011). Cognitive presence is one of the core 
components of the Community of Inquiry (Col), a theoretical framework which conceptualizes 
online learning processes as consisting of three interdependent presences: social, teaching, and 
cognitive presence (Garrison et al., 2001). Specifically, cognitive presence includes four 
categories: (1) triggering event, (2) exploration, (3) integration, and (4) resolution. The triggering 
event indicates “a state of dissonance” or “feeling of unease resulting from an experience” 
(Garrison, Anderson, & Archer, 1999, p. 98). Exploration relates to investigating the divergence 
of ideas. It includes searching for new infonnation, knowledge, and alternatives to address a 
problem. Integration focuses on the convergence of different ideas. Finally, resolution focuses on 
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application of solutions in the real world and defending those solutions (Akyol & Garrison, 
2011; Garrison et al, 1999). 

Previous studies exploring cognitive presence found that cognitive presence has been 
associated with perceived and actual learning outcomes (Akyol & Garrison, 2008; Joo, Lim & 
Kim, 2011). Other studies have reported that exploration level was the most coded indicator 
while the rate of integration and resolution were relatively small (Garrison et al., 2001; Kanuka, 
Rourke, & Laflamme, 2007). These earlier studies indicate that students are not likely to reach a 
higher level of cognitive presence in online discussions (Kanuka et al., 2007). However, Swan, 
Garrison, and Richardson (2009) proposed that the integration and resolution stages might 
actually appear in students’ final assignments rather than in online discussions as students 
demonstrate their learning and knowledge construction from an entire course. Swan et al. (2009) 
and Akyol and Garrison (2008, 2011) found that integration was the mostly widely coded level 
in students’ online discussions facilitated by students, when the course instructor modeled how to 
facilitate discussions in an effective way. 

Summarizing the research, we have found inconsistencies in the distribution of categories 
of cognitive presence, which may be due in part to other factors, including differences in 
instructional strategies, discussion topic choices, purpose of discussion boards (to introduce 
topics or demonstrate knowledge), and course design. It could also be related to the simple fact 
that one would expect more exploration, a little less integration because these posts build on two 
or more exploratory ones, and even fewer examples of resolution as these indicate a final 
decision reached regarding a problem or issue raised. 


Role Assignments as an Instructional Strategy 

Research has suggested that using scripted roles has a positive influence on learning 
within online discussions by improving the responsibility of each member and interdependence 
among group members (Brush, 1998; Hare, 1994; Mudrack & Farrell, 1995; Strijbos, Martens, 
Jochems, & Broers, 2004). Types of scripted or assigned roles have varied in the literature. For 
example, De Wever, Van Keer, Schellens, & Valcke (2009) suggested five roles: (1) starter, to 
get the discussion started and give new points to promote discussions; (2) moderator, to provide 
relevant prompts and questions and monitor the discussion; (3) theoretician, to provide 
appropriate theories to deepen the discussion; (4) source searcher, to provide external reading 
sources or materials about the topic of discussion, and (5) summarizer, to summarize the postings 
on discussion boards. 

Other studies also examined the impact of a particular role on student learning. For 
example, Schellens, Van Keer, and Valcke (2005) found that students who played a summarizer 
role were able to achieve a higher level of knowledge construction. In addition, Xie et al. (2014) 
reported on the positive effects of assigning a role. They found that assigning a moderator role 
could promote greater student participation. Students who were assigned the moderator role 
posted more messages and stayed logged in longer than students who were not assigned a 
moderator role (Xie et al., 2014). Wise and Chiu (2012) focused on the influence of assigning 
various roles on the levels of knowledge construction including: (1) sharing information; (2) 
exploring dissonance; (3) negotiating meaning; (4) testing and modifying; and (5) agreeing and 


Online Learning - Volume 20 Issue 4 - December 2016 


37 



Exploring the Effect of Scripted Roles on Cognitive Presence in Asynchronous Online Discussions 


applying by using the model developed by Gunawardena, Lowe, and Anderson (1997). They 
found that students who were assigned a synthesizer or a wrapper role contributed more and 
higher quality posts than other assigned roles. 

Together, the results of previous research indicate that assigning scripted roles may have 
positive effects on students’ level of cognitive presence and the effects of scripted role 
assignments are likely to be varied according to the types of scripted roles. 

Roles of Question Prompts in Facilitating Cognitive Presence 

Researchers have explored the effects of different types of question prompts on students’ 
cognitive and critical thinking abilities in asynchronous online discussions (deNoyelles et ah, 
2014; Richardson et ah, 2012). For example, Ertmer, Sadaf, and Ertmer (2011) examined the 
relations between different types of question prompts and the levels of critical thinking in 
students’ discussion postings based on Bloom’s taxonomy (knowledge, comprehension, 
application, analysis, evaluation, and synthesis). They found that higher level questions are likely 
to lead to responses which are categorized at higher levels of Bloom’s taxonomy while lower 
level questions led to lower level responses such as knowledge and comprehension (Ertmer et al., 
2011). Furthermore, de Noyelles et al. (2014) suggested that the use of structured design in 
discussion prompts can be an effective strategy to improve students’ cognitive presence. They 
suggested three different types of prompts that can improve students’ higher level of cognitive 
presence than other traditional question-answer prompts (deNoyelles et al., 2014): (1) problem- 
based prompts—asking students to find solutions through discussion; (2) project-based 
prompts—asking students to create an artifact through participating in a problem solution 
process; and (3) debate prompts—choosing a position and making an argument to persuade other 
people who have different positions. 

Richardson and Ice (2010) explored three different types of questions including case- 
based, debate, and open-ended questions. They found that integration indicators were most 
frequently found in all three discussion types, followed by exploration indicators. They also 
found that the rate of resolution indicators was higher in the case-based discussions than in the 
other two questions types. The rate of resolution was very low compared with indicators for 
other cognitive presence levels (Richardson & Ice, 2010). To summarize, previous research 
findings revealed that the types of question prompts play an important role and have an impact 
on the levels of cognitive presence developed in online discussions. 


Methods 

A quantitative content analysis research design was used to explore the effect of role 
assignment as an effective instructional strategy to promote the development of cognitive 
presence among students in asynchronous online discussions (Berelson, 1952). Content analysis, 
“a research technique for the objective, systematic, and quantitative description of the manifest 
content of communication” (Berelson, 1952, p. 18), has been widely used to analyze discussion 
postings in previous research (Garrison et al., 2001; Gorsky, Caspi, Blau, Vine, & Billet, 2011; 
Strijbos et al., 2006). 

In this study, students in an online course “Introduction to Nutrition” played one of the 
assigned roles (starter, skeptic, or wrapper) when they participated in asynchronous online 
threaded discussions. Students rotated through the roles during the course with some students not 
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assigned a role but still required to participate; the exception was students enrolled in Fall 2013 
when all students were assigned a role. Therefore, in this study, we labeled “with the role 
assignment” when students were assigned a role during weekly online discussion and “without a 
role assignment” when students were not assigned a role during the same weekly online 
discussion but were still participating. 

The collected discussion posts (n=2166) across all three semesters were coded using the 
cognitive presence indicators including triggering event, exploration, integration, and resolution. 
Then we transfonned the qualitative data into frequencies. The frequencies were compared 
between the discussion posts generated by students “with the role assignment” and “without a 
role assignment.” The frequencies were analyzed to identify any associations between students’ 
level of cognitive presence and the types of assigned roles, no assigned roles, across all three 
semesters. 


Participants 

Three semesters of data included Fall 2013, Spring 2014, and Summer 2014. Across 
these semesters, one student was excused from discussion assignments. Otherwise, all students 
(n = 139) were included in the study. Each discussion occurred over the course of one week, to 
coincide with the weekly course module structure. The frequency of discussions and allotment of 
scores varied by semester (see Table 1). Participation in the online discussions was required and 
evaluated every semester. 


Table 1 

NUTR 295 Section Characteristics by Semester 


Semesters 

Students 

Number 

Groups 

of Number of 

Online 
Discussions 

Weeks of 

Online 

Discussions 

% of Final 
Grade from 

Discussion 

Fall 2013 (15-week 

n=26 

4 groups 

3 discussions 

Weeks 2, 6, & 

7% 

semester) 




13 


Spring 2014 (15- 

n=38 

6 groups 

6 discussions 

Weeks 2, 3, 4, 

18% 

week semester) 




6, 8, & 13 


Summer 2014 (6- 

n=76 (2 

15 groups 

4 discussions 

Weeks 2, 3, 4, 

25% 

week semester) 

sections) 



& 5 



Students were randomly assigned to small groups of between five and seven students by 
using the Blackboard “random assign” function, regardless of overall class size. Across the three 
semesters, a total of 139 students participated in 25 groups, with a mean group size of 5.6. In 
each semester, seniors predominated among registered students, representing 41% of the students 
over the three semesters. Sophomores and juniors filled 21% and 22% of the seats, respectively. 
Remaining seats were divided among freshmen (5%), graduate students (8%) mostly fulfilling a 
pre-requisite for entry into the Nutrition MS program, and non-degree seeking students (3%). 
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The researchers for this study were the course instructor, an instructional designer who 
helped design the role assignments in this course, and a research assistant who helped code 
discussions. All were involved in the research process as well. 

Context 

This course introduces students to nutrition as a scientific discipline, providing a working 
knowledge of the food sources and bodily functions related to nutrients, the current guidelines 
for a health-promoting diet, and strategies for detecting nutritional misinfonnation. The course is 
a required course for several pre-health undergraduate degrees and the Nutrition minor, and is 
approved as a natural science core course by the university. It attracts a broad variety of students 
across the university, but a high concentration of pre-health undergraduate majors is present, and 
a predominance of female students is typical of students enrolled in nutrition courses. 

Discussions occurred entirely asynchronously in the Blackboard Learning Management 
System, over the course of one week, from Wednesday through the following Tuesday. Each 
group of 5-7 students was assigned a private discussion board and could not see the other groups’ 
conversations. Three special roles were assigned randomly for each discussion at the beginning 
of the semester, via a table provided to students. Thus, students were aware of their upcoming 
roles in advance. Assigned roles were as follows: (1) the s tarter was responsible for reading the 
week’s discussion prompt, starting a conversation toward fulfilling the requirements of the 
prompt, and suggesting a structure/schedule of the group’s response; (2) the skeptic had to 
intellectually challenge the thoughts and assertions of group mates throughout the week (e.g., 
asked questions); (3) the w rapper summarized the group’s discussion at week’s end; and (4) no 
assigned role participated in the discussion without a specific role. The three roles seemed 
sufficient to start the online discussion, carry it forward, and end the weekly conversation. 

At the beginning of each discussion week, a discussion prompt was provided to the group 
in a dedicated forum, such as Week 3 Online Discussion. Discussion prompts contained: (1) a 
description of the topic for the week; (2) a short background reading and video; and (3) an 
activity to accomplish by the end of the week, such as to answer a set of questions or design a 
hypothetical research study. Each group received the same discussion prompts within a semester, 
but prompts varied somewhat between semesters. Table 2 presents the types of questions with 
summarized examples. 

Table 2 

Examples of the Online Discussion Questions by the Type of Questions 


Types of Examples Semesters 

Questions _ 

Article Summary As a group, you’ll discuss the merits and drawbacks of the Summer 2014, 
(AS) article, the methods the authors used, and what the results OD#l 

of the study mean in the real-world. What is the author's 
rationale for conducting the study? Do you agree that this 
is worth studying? Why or why not? Describe the research 
methodology used in the study. Did these methods allow 
the authors to answer their research question(s)? In your 
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Types of 

Questions_ 


Debate (D) 


Design 

Intervention (DI) 


Design Study 
(DS) 


Scenario-Based 

(SB) 


Search and 
Match (SM) 


Examples 


own words, summarize the results of the study. In your 
opinion, what is the * real * impact of these results? (Does 
it mean anything to you or your family?) What are some 
limitations to this research article that were not stated by 
the authors? 

This debate will require you to find a scientific reference 
and a non-scientific reference to support your assigned 
topic. You'll present the infonnation to your peers, then 
will ask and answer questions from your peers who were 
assigned an opposing topic. 

As a group, select an age-group from the four choices 
below. As a group, name at least 3 interventions that we 
could do as a society to promote physical activity in your 
chosen age group. (For example, we could support the 
construction of bike paths and bike lanes to promote 
physical activity during one's commute to campus. This 
would obviously target the age group of college students.) 
As a group, choose one of your suggested interventions 
and describe how you would scientifically test if your 
intervention is successful in reducing or preventing 
obesity. 

The objective of this discussion is to continue practicing 
the scientific method and get additional practice at 
identifying the strengths and weaknesses in a research 
study design. Describe a basic research study that tests 
your hypothesis. Describe at least 2 strengths of your 
proposed research study. Describe at least 2 weaknesses of 
your proposed research study. 

You are a physician interested in whether increasing 
dietary selenium will decrease prostate cancer. You are 
recruiting 20-50 year-old males for your study. Half will 
be on a usual diet with 100% of the Recommended Dietary 
Allowance (RDA) per day and the other half will be on the 
experimental diet with 300% of the RDA per day (a level 
considered safe). You will measure the incidence of 
prostate cancer at the end of 5 years. What type of study is 
this (according to the list on page 3)? What aspects of the 
study design tell you this? Write a potential hypothesis for 
the study. 

Search the internet for nutrition products’ websites which 
you might consider to contain nutrition misinformation. 
Your team's task for the week is to identify examples of 
*each* of the 11 Earmarks of Nutrition Quackery on 
nutrition products' websites. For each eannark, you must 


Semesters 


Summer 2014, 
OD#4 


Dll (Fall 2013, 
OD#3) 

DI2 (Spring 
2014, OD#6) 


DS 1 (Spring 
2014, OD#2, 

OD#3) 

DS2 (Summer 
2014, OD#2) 


SB1 (Fall 

2013, OD#l) 
SB2 (Spring 

2014, OD#l) 


SMI (Fall 

2013, OD#2) 
SM2 (Spring 

2014, OD#4, 
OD#5) 
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Types of Examples Semesters 

Questions _ 

provide a link to the website and provide a description of SM3 (Summer 
where the eannark appears on the website. 2014, OD#3) 


Students were provided with a grading rubric in advance, which allotted points for 
timeliness, quantity, pertinence of post content, demonstration of knowledge and assigned 
readings, and completion of the assigned special role. Examples of discussion posts of varying 
quality and role were also provided. A minimum of three total posts were required of each 
student each week, regardless of assigned role: one post by Saturday and two subsequent posts 
by Tuesday. The instructor limited feedback during discussion, except to provide guidance on 
structure. After each discussion, feedback and scores were provided via the electronic rubric. 


Data Collection and Analysis 

Qualitative data were collected from students’ discussion posts (n=2166) across all three 
semesters in Fall 2013 (n=285), Spring 2014 (n=878), and Summer 2014 (n=1003). Students’ 
discussion posts were coded and categorized according to the Practical Inquiry Model (Garrison 
et ah, 2001). The discussion posts were determined as the unit of the analysis corresponding to 
what one student posted into one thread in the discussion boards. The posts were marked in each 
thread for coding analysis; the length and the content of each discussion post was decided by the 
students enrolled in the course in Fall 2013, Spring 2014, and Summer 2014. The codes were 
assigned exclusively, that is, each post could only be assigned one code. For example, we coded 
the post as a “triggering event” when it presented the sense of puzzlement and recognized the 
problem; “exploration” when it demonstrated infonnation exchange and suggestions for 
consideration; “integration” when it created solutions and connected ideas and/or synthesis; and 
“resolution” when it tested and defended a solution (see Table 1). The posts with organizational 
questions, clarification issues, appreciation, and confirmation were not coded and were excluded 
from this study. 

The coded data were tabulated into frequencies for the descriptive statistical analysis, the 
Chi-square test to analyze differences between roles and cognitive presence, the Mann-Whitney 
U test to analyze differences between semesters, and a Kruskal-Wallis test to compare 
differences between the three semesters. 

Reliability and Validity 

There were two coders (the instructional designer who was also a researcher in this study 
and a graduate student) for each online discussion to establish the reliability and validity of the 
coding process. An inter-rater reliability of 95% was reached. The inter-rater reliability was 
computed using the formula where reliability is equal to the number of agreements divided by 
the sum of agreements and disagreements (Miles & Huberman, 1994). In the content analysis 
approach, the training of the coders is important to establish improved reliability (De Wever, 
Schellens, Valcke, & Van Keer, 2006). The researchers, who were also coders in this study, were 
provided with the categories with which to code the posts and the tables with indicators and 
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descriptions of socio-cognitive processes (see Table 2). A brief coding training meeting was also 
held to discuss the coding procedures before starting the individual coding. 

Each coder independently coded posts. After coders finished their own coding, the coded 
and categorized discussion comments were compared with attention to the levels of cognitive 
presence outlined by Garrison et al. (2001). Initially, the researchers created a comparative table, 
arranging codes according to the levels of cognitive presence in order to find evidence related to 
the research questions. Following this, each researcher examined the data to detennine 
differences across the three assigned roles and no roles across three semesters. Researchers also 
discussed the results to clarify individual interpretations to come to consensus. 

Table 3 

The Levels of Cognitive Presence bv Indicators and Socio-cognitive Processes (Garrison et al., 

2001 ) 


Category _ 

Triggering events 


Exploration 


Integration 


Indicator 

Recognizing the problem 
Sense of puzzlement 


Divergence - within the 
online community 
Divergence - within a single 
message 

Infonnation exchange 


Suggestion for consideration 


Brainstonning 


Leaps to conclusions 
Convergence - among group 
members 


Convergence - within a 


Socio-cognitive processes 
Presenting background 
infonnation that culminates 
in a question 

Asking questions, messages 
that take discussion in new 
direction 

Unsubstantial contradiction 
of previous ideas 
Many different ideas/themes 
presented in one message 
Personal 

narrative/ descriptions/facts 
(not used as evidence to 
support a conclusion) 

Author explicitly 
characterizes message as 
exploration - e.g., “Does that 
seem about right?” or “Am I 
way off the mark?” 

Adds to established points 
but does not systematically 
defend/j ustify/develop 
addition 

Offers unsupported opinions 
Reference to previous 
message followed by 
substantiated agreement, e.g., 
“I agree because....” 

Building on, adding to others’ 
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Resolution 


*reprinted with permission 


single message 
Connecting ideas, synthesis 

Creating solutions 


Vicarious application to 
world 

Testing solutions 
Defending solution 


ideas 

Justified, developed, 
defensible, yet tentative 
hypotheses 

Integrating infonnation from 
various sources - textbook, 
articles, personal experience 
Explicit characterization of 
message as a solution by 
participant 


real None 
Coded 


Results and Discussion 
Research Question #1 

To answer the main research question on what level of cognitive presence was achieved 
by students’assigned roles in asynchronous online group discussions; the analysis of the percent 
of posts generated by role assigmnent during assigned weeks and without role assignment in 
subsequent weeks was run. Similar to Akyol and Garrison’s (2008, 2011) findings, the majority 
of cognitive presence indicators occurred at exploration and integration levels for the weeks 
when students performed the role (24.06% and 70.96%) and afterwards (19.14% and 77.95%) 
when they did not perform the role (Figure 1). However, triggering events and resolution were 
the least coded levels of cognitive presence for both conditions during the weeks when students 
performed the role (3.23% and 1.75%) and afterwards (0.61% and 2.30%) when they did not 
perfonn the role. 


90% 


80% 

70% 


60% 

50% 


40% 

30% 


20 % 

10 % 




Triggering 

Exploration 

Integration 

Resolution 

1 ■ With Role 

3.23% 

24.06% 

70.96% 

1.75% 

■ Without Role 

0.61% 

19.14% 

77.95% 

2.30% 


Figure 1. Percent of posts by the levels of cognitive presence with role assignment and without 
role assignment for all semesters. 
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As the descriptive statistical analysis did not find differences between the levels of 
cognitive presence with and without role assignment, a chi-square test was conducted to examine 
associations between the levels of cognitive presence in the weeks when students perfonned the 
assigned role vs. in the weeks when the same students did not perfonn any role. The results of a 
chi-square test indicated a significant difference in the levels of cognitive presence between 
weeks when students perfonned the role and the weeks when they did not perfonn the role (y 2 (3, 
A=2199) = 21.57,/? <.05). 

To analyze what type of role (starter, skeptic, or wrapper) was the most effective, the 
percent of posts generated by each role were analyzed. The results of descriptive statistics 
revealed that all three roles were effective. The highest percent of posts for all three roles 
occuned at exploration and integration levels (Figure 2). Similarly, triggering and resolution 
were the least coded levels for all three roles. To examine differences between the types of roles 
and the levels of cognitive presence, a chi-square test was conducted. The results of a chi-square 
test indicated there was a significant difference between the types of roles and the levels of 
cognitive presence (y 2 (6, A=1546) = 16.83,/? <.05). 
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Figure 2. Percentages by the levels of cognitive presence and by three roles across fall 2013, 
spring 2014, and summer 2014 (combined). 


Research Question #2 

To answer the second research question as to whether the levels of students’ cognitive 
presence were different between when the asynchronous online group discussions occurred 
during the 5-week intensive summer course in Summer 2014 versus the 15-week regular 
semesters in Fall 2013 and Spring 2014, a Kruskal-Wallis test was run. The analysis did not find 
any statistically significant difference at triggering and resolution levels of cognitive presence 
when students participated in a 5-week intensive summer course or a 15-week regular semester 
in fall and spring. However, there were significant differences at exploration level (y 2 (2, n= 314) 
= 7.84,/? =.02) and integration level (y 2 (2, n=551) =23.69, p=. 00) across three semesters. The 
online discussion posts in Fall 2013 had the highest median scores at triggering level (Md = 1.50) 
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and exploration level (Md = 2.00) than the other two semesters, which both had median scores of 
1.0 at both levels of cognitive presence. This might be explained that all students in Fall 2013 
were assigned a role which means they all played one of the assigned duties compared to 
students in Spring 2014 and Summer 2014. 


Research Question #3 

The descriptive statistical analysis was run to compare percent of posts for each type of 
question for all three semesters combined. Figure 3 shows that percent of posts at the exploration 
and integration levels were much higher than percent of posts at the triggering level and the 
resolution level. The highest percent of posts at resolution level (7.61%) occurred for the 
question when students were asked to design their own intervention. However, there were no 
posts found at the resolution level for the questions on the article summary and debates. Both 
questions were used in Summer, 2014. The question asking the students to summarize an article 
was used in the first online discussion where students were asked to summarize the results of the 
study and provide their opinion on the impact of the results. At the same time, this type of 
question had the highest percent of posts at the exploration level (29.22%). The indicators for the 
exploration level are different within the online community or within a single message, 
information exchange, and brainstonning. In this discussion, students mostly shared infonnation, 
discussed the article, argued, or brainstormed their ideas. The debate question took place in the 
last online discussion. The debate question asked students to justify their topic and ask opposing 
sub-groups at least one question. That may be why this type of question received the highest 
percent of posts (10.56%) at the triggering event level. One of the socio-cognitive processes at 
the triggering event level is question asking. Thus, these findings showed that the types of 
questions have a direct relationship to the level of cognitive presence. For example, when 
students were asked to brainstonn and summarize ideas, their level of cognitive presence usually 
stayed at the exploration level. These findings agree with previous research that higher level 
questions are likely to lead to responses which are categorized at higher levels while lower level 
questions, such as the article summary in this study, led to lower level responses like knowledge 
and comprehension (Ertmer et ah, 2011). 
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Figure 3. Percent of posts at the levels of cognitive presence as an effect of the type of question. 
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Further, we examined whether there were differences between the three semesters (fall 
2013, spring 2014, and summer 2014) in the levels of cognitive presence by the type of 
questions. To reveal the differences, a Mann-Whitney U test was applied. Table 4 presents the 
results where the differences were found. 

Table 4 

Comparison across semesters by the type of questions 


Types of 
Question _ 

Design 

Intervention 

(DI) 


Design Study 
(DS) 


Scenario-Based 

(SB) 


Analysis 


The two samples were significantly different (p<.05) 
for the levels of cognitive presence at: 

• Exploration for fall 2013 (Md=2.00, n=19) and 
spring 2014 (Md=1.00, n=8); U= 28.000, z=- 
2.83, p=. 005, r=. 5). 

• Integration for fall 2013 (Md=2.00, n=19) and 
spring 2014 (Md=3.00, n=35); U= 132.5, z=- 
3.74, p=. 000, r=. 5). 

The two samples were significantly different (p<.05) 
for the levels of cognitive presence at integration for 
spring 2014 (Md=3.00; n=72) and summer 2014 
(Md=2.00; n=66); U=1916.000, z=-2.034, ^=.04, 
r=. 2). 

The two samples did not differ significantly for any 
of the four levels of cognitive presence. 


Search and The two samples did not differ significantly for any 
Match (SM) of the four levels of cognitive presence. 


The two samples were significantly different (p<.05) 
for the levels of cognitive presence at integration for 
spring 2014 (Md=3.00; n=70) and summer 2014 
(Md=2.00; n=68); U=1474.500, z=-3.94, ^=.000, 
7 ^. 3 ). 

The two samples were significantly different (p<.05) 
for the levels of cognitive presence at integration for 
fall 2013 (Md=3.00; n=24) and summer 2014 
(Md=2.00; n=68); U=490.500, z=-2.97, ^=.003, 
r=. 3). 

*OD - Online Discussions 


Semesters 


Dll (Fall 2013, 
OD#3) 

DI2 (Spring 2014, 
OD#6) 


DS1 (Spring 2014, 
OD#2, OD#3) 

DS2 (Summer 2014, 
OD#2) 

SB1 (Fall 2013, 
OD#l) 

SB2 (Spring 2014, 
OD#l) 

SMI (Fall 2013, 
OD#2) 

SM2 (Spring 2014, 
OD#4, OD#5) 

SM2 (Spring 2014, 
OD#4, OD#5) 

SM3 (Summer 2014, 
OD#3) 

SMI (Fall 2013, 
OD#2) 

SM3 (Summer 2014, 
OD#3) 


Students’ levels of cognitive presence differed when they were asked to design their own 
intervention. This type of question was used in Fall 2013 and Spring 2014 in the last online 
discussions. Students in both semesters answered the question differently. Figure 4 shows that 
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students’ posts in Spring 2014 mostly stayed at the integration level (84.44%) while posts in Fall 
2013 were evenly distributed between the exploration (47.56%) and the integration (47.56%) 
levels. This difference might be explained by the fact that all students in Fall 2013 were assigned 
a role while students in Spring 2014 were either in the condition “with a role assignment” or 
“without a role assignment.” It seems that playing an assigned and required role, such as skeptic, 
might have prevented students from posting more at the integration level. Students taking the 
skeptic role usually provoked discussion by asking questions and brainstonning ideas. The 
findings of this study are similar to what Richardson and Ice (2010) found regarding case-based 
discussions, in our case designing a new intervention, having a higher percentage of resolution 
indicators than other types of questions in this study because the task is asking for resolution. 



Figure 4. Comparison of Design Intervention (DI) question by percentages for the levels of 
cognitive presence. 


Next, the students’ posts were different between semesters when they were asked to 
design a study (Figure 5). This question was used in two discussions during Spring 2014 but in 
only one discussion in Summer 2014. During Spring 2014, students were asked to design their 
own study in online discussion #2 and they also discussed ethical considerations including 
consent fonn, recruitment, privacy, and confidentiality for their designed study in online 
discussion #3. Students who had two online discussions in Spring 2014 posted more messages at 
the integration level. Their posts showed a higher percentage of convergence among group 
members; they connected ideas or synthesized ideas, and/or created their own solutions more 
often than students in Summer 2014 who had only one discussion in which to design their study. 
It seems that having an extended time to discuss a research design can help students create more 
convergence and/or synthesis or ideas among group members instead of only brainstonning or 
exchanging ideas at exploration level. 


Online Learning - Volume 20 Issue 4 - December 2016 


48 



Exploring the Effect of Scripted Roles on Cognitive Presence in Asynchronous Online Discussions 



Figure 5. Comparison of Design Study (DS) question by percentages for the levels of cognitive 
presence. 

The question that asked students to search for online resources on nutrition products to 
match them with the identified a supplied list of criteria was used across all three semesters. 
Across all three semesters students answered this type of question differently. Figure 6 shows 
that fewer of the students’ posts in Summer 2014 (SM3) were at the integration level (74.79%) 
as compared with Fall 2013 (SMI) (86.31%) and Spring 2014 (SM2) (86.99%). This finding 
suggests that students in 15-week classes in fall and spring posted more messages at the 
integration level than students who were enrolled in an intensive five-week summer course. 



Figure 6. Comparison of Search and Match (SM) question by percentages for the levels of 
cognitive presence. 
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Implications and Future Research 

This study found evidence that role assignment can be an effective instructional strategy 
to facilitate higher level learning when implemented in asynchronous online discussions. The 
results of this study found a significant difference between two conditions when students were 
assigned a role versus when they were not. Contrary to previous studies’ findings, the majority of 
students’ posts in this study remained at the integration level across all three semesters. This 
difference can be explained by the nature of this course in which students worked on their own 
research design or intervention. This difference can also be explained by the type of questions 
and the design of online discussions in which the discussion prompts contained: (1) a description 
of the topic for the week; (2) a short background reading and video; and (3) an activity to 
accomplish by the end of the week, e.g. answer a set of questions or design a hypothetical 
research study. The online discussions were also accompanied by the assessment rubric and 
examples for the quality of online postings. Therefore, online instructors, instructional designers, 
and other practitioners are encouraged to implement role assignment as an effective instructional 
strategy. However, role assignment should be supported by the type of questions that help 
students achieve higher levels of cognitive presence. Examples of questions that can support 
higher levels of cognitive presence could be those that focus on designing an intervention or 
search and match. Moreover, online discussions should also be supported by extra resources such 
as assessment rubrics and examples of posts. 

Future research should include a comparison of online discussions in which role 
assignment as a treatment can be compared to a control group to find evidence of the 
effectiveness of role assignment to facilitate cognitive presence among students. More research 
across different disciplines and different role assignments is also needed. 


Conclusion 

The findings of this study revealed the effect of scripted role assignments on the levels of 
cognitive presence in asynchronous online discussions, in a natural science course. Findings 
revealed that the majority of students’ posts were at a higher level of cognitive presence— 
integration. Unlike findings from previous studies that found that the majority of students’ online 
postings are usually at a lower level of cognitive presence—exploration, this study’s findings are 
promising. In this study, the majority of discussion posts were at integration level across all three 
semesters. However, more research is needed to explore the differences between a control group 
with no role assignment and an experimental group with role assignment. 
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